Stereo night vision system for vehicles

ABSTRACT

The invention relates to a night vision system for vehicles. According to the invention, the road in front of the vehicle is stereoscopically recorded via two night vision-capable cameras, e.g. infrared or low-light-level cameras that are mounted in the front of the vehicle at a distance from one another. Corresponding stereoscopic image display devices permit the driver to observe the road in front of the vehicle in three-dimensions. Objects located in front of the vehicle can be detected in critical areas by means of additional image evaluating devices and can be differently accentuated or marked in the image representation. The night vision system can be designed to also permit a two-dimensional display of images or data alternatively or in conjunction with 3D representation. In an enhanced embodiment, head movements and/or the line of sight of the driver are/is detected and used for a corresponding tracking of the image display.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to night vision systems in which images arerecorded by cameras and subsequently conditioned electronically. Theinvention relates in particular to such systems that are suitable foruse in vehicles on the basis of their properties.

2. Related Art of the Invention

The fact that the accident rate is substantially higher at night thanduring the day is substantiated by accident statistics. The reason isthat the auto driver primarily makes use of the sense of sight in orderto control a vehicle and estimate danger. At night, there is much lessinformation about the surroundings that can be gathered by sight; inparticular unexpected laterally occurring dangers are perceived later.

However, the speeds being driven and reaction times they require remainessentially the same.

There is a range of technical systems in vehicles that support thedriver in perceiving the driving path lying ahead at night. Theseconsist generally of a night vision capable camera (with additionalillumination device if appropriate) and reproduction of the recordedscene for the driver. Thus, for example, use has been made of thermalimaging cameras (BMW) or it has been proposed to use UV illuminatingdevices in conjunction with fluorescing marking elements (Volvo).

A stereoscopic arrangement of infrared cameras has been described inIEEE Intelligent Vehicles Symposium (Proceedings IV-2001, May 13-17,2001 Tokyo Japan). In this system, the area of approximately 30 to 80meters lying in the driving direction is sensed simultaneously by 2infrared cameras that are fitted in the front region of the vehicle andare at a spacing of 360 mm from one another. Objects in the drivingdirection (for example passers-by) are detected by comparing thestereoscopic images, as is their distance from the vehicle.

The video image picked up by cameras is conditioned and reproduced invarious ways for the driver in the known systems. Customary, forexample, are monitors in the dashboard area or head-up displays that arefitted in the driver's visual range. Reflection into the bottom of thewindshield can also be performed as an alternative.

With head-up displays in which the driver looks through in the drivingdirection, it is a problem to bring the camera image precisely intocongruence with the scene directly perceived. Again, it is difficult toensure a display of the camera image that does not in some circumstancescover up details of the real scene situated there behind, owing toexcessively high levels of intensity. When monitors are arrangedseparately, for example in the console or in the region of the bottom ofthe windshield, in order to pick up the information displayed the drivermust in each case look away from the driving direction—in a way rathersimilar to looking away when using an interior or exterior mirror.

The known systems therefore have the disadvantage that the driver'sattentiveness can be diverted. So that a night vision system increasessafety and does not additionally divert the driver, the latter mustgrasp and interpret the reproduced scene in a particularly simple andintuitive way.

SUMMARY OF THE INVENTION

Starting from this prior art, it is the object of the present inventionto develop an improved system that largely overcomes the saiddisadvantages and has additional advantages.

This object is achieved in the case of a night vision system having thefeatures of the preamble of claim 1 by means of the characterizingfeatures of claim 1. Further details of the invention and advantages ofvarious embodiments follow from the features of the subclaims.

The inventive system and corresponding apparatuses are described belowwith the aid of preferred embodiments.

The present invention serves the purpose of presenting the driver withan image of the scene in front in a way that can be grasped as quicklyas possible in conjunction with minimal diversion. Use is made for thispurpose of the fact that a person can perceive a depth, offset of thescene, given with stereoscopically presented images. In the case of thesystem according to the invention, two conventional night vision capablecameras (e.g. infrared or residual light cameras) are used in a stereoarrangement for image pickup and the resulting scene is displayed to thedriver by means of stereoscopic reproduction devices. If the base widthof the cameras is selected to be greater than the human eye base, thestereo effect is amplified and the depth offset can be detected evenmore effectively by the driver. Thus, when three times the eye base isselected it is still possible to perceive a depth offset of up toapproximately 70 m. Owing to the three-dimensional reproduction, thedriver can grasp the geometrical arrangement of objects in the drivingdirection intuitively, and therefore quickly and with minimal diversion.

Various devices can be used for reproducing the stereo images. Thus, ina first embodiment the stereo images can be displayed on a conventionalmonitor or LCD display. Alternatively, reflection onto the windshield isalso possible. In this case, the position is preferably selected suchthat in the event of looking in the driving direction the reproductionis superimposed on the real field of view. Other positions (consol,dashboard region, bottom of the windshield) are likewise possible, butrequire a change of direction of view during driving.

The stereo images can, for example, be split up via optical aids thatare assigned directly to the viewer. Customary here are, for example,special spectacles that function according to the anaglyph method(color), the method of temporal stereography (shutter) or the principleof polarization stereography (polarization filter). Combinations of suchmethods are also possible. Since these methods are based on theexclusion principle (it is only the respectively assigned images orimage components that can be viewed separately for each eye), theyreduce the quantity of light and therefore impede the direct view to theoutside in some circumstances.

Other methods that proceed according to the autostereoscopic principleachieve a three-dimensional perception without the need for the viewerto wear optical aids (free viewing). A split up of the stereo images canbe based in this case on diffraction-based elements, for examplediffractive optical elements (DOEs, gratings) or holographic elements(HOE) that emit the stereoscopic information, shown on a display ordisplayed by projection, in various directions. The image can also besplit up via refraction-based elements, for example arrays oflongitudinal prisms, microlens systems, cylindrical lens arrays or fieldlenses that reflect the various stereoscopic image componentsdifferently in cooperation with the driving of the display or projector.Also suitable are reflecting elements that direct stereoscopicinformation to the viewer in a directionally selective fashion.

In another embodiment, the propagation of light is prevented in specificdirections via barrier grids or color masks for the purpose of splittingup the stereo images spatially (exclusion method or covering method).

As an alternative, the stereo images can be split up via structuredillumination (parallax illumination) of displays (e.g. LED displays),different information being emitted in different directions by a singlestructural unit of the overall illumination.

Instead of a simultaneous spatial separation of the two stereo imagecomponents, the stereo images can also be split up by means oftime-division multiplexing (moving-slit method) in the case of which thestereo information is split up sequentially so quickly in the timedomain into the respective directions of view for various image contentsthat the viewer gains the impression of a complete stereo image.

An impression of depth and effect of depth can also be achieved by meansof a 2% D display, in which two or more image planes are arranged onebehind another. In this case, the image information is displayed byreflecting partial images into the individual image planes arranged onebehind the other, for example via semitransparent mirrors, it beingpossible to back up the image display by suitable weighting of contrastand intensity.

Another modern principle for three-dimensional image reproduction islikewise suitable for use in the proposed night vision system. Thisprinciple is based on the electroholographic method in which a reducedhologram function is compiled from the stereographic information and isdisplayed by means of scanning laser systems and electrooptic modulators(principle of MIT, Massachusetts Institute of Technology).

Further methods are known for 3D image reproduction, and are suitablefor the night vision system according to the invention. Thus, thestereoscopic information can be transferred onto volumetric displays(for example periodically moving displays, specially shaped, rotating ormoving projection screens, nonlinear effects in volumetric media suchas, for example, 2-photon fluorescence excited by space-divisionmultiplexed laser beams). Again, the stereo images can be split up viatwo or more projectors whose emitted images are perceived in anangularly selective fashion as real or virtual images (for examplestereoscopic head-up displays with two projectors).

Combinations of the various methods set forth here can also be used forthe night vision system according to the invention.

In an extended embodiment, means are provided for detecting the driver'shead and/or eye position (observation of the driver's direction ofview). It is possible for this purpose to use, for example, one or morecameras in the interior of the vehicle with image evaluationappropriately connected downstream but the eye position can also bedetermined (for example by triangulation) with the aid of infrared orultrasonic probes. The data thus obtained on the current direction ofview can be used to drive the above-named devices for thethree-dimensional image reproduction in such a way that the stereoimages are tracked as a function of the head or pupil position (forexample by the displacement of masks, light sources or the lightmodulator) so that the stereoscopic impression is maintained even whenthe head position is varied. It is possible in this case to interpolatevarious directions of view and to feed the stereo images to the viewerin a new direction of view given a changed head/eye position, thusproducing a movement parallax.

If the image reproduction device is designed such that there is notracking as a function of the eye position, the image can be split upspatially in such a way that the stereo impression arises for aprescribed (for example central) head position, and that upon adisplacement of the head from the defined position both eyes receiveidentical images and so only a purely two dimensional display takesplace.

In a particular embodiment, the different 3D images are selectivelyprojected via appropriate light sources onto the respective retina ofthe viewer. It is possible for this purpose to make use, for example, oflaser systems or special LED projectors.

In the case of all the three-dimensional image reproduction devices setforth, the objects detected by the camera system in critical distanceranges of the driving path lying ahead can additionally be opticallymarked by electronic conditioning in order to support the driver (forexample by coloring, contrasting, flashing, etc.). It is also possiblefor such objects to be selectively displaced into other spatial planesof the 3D reproduction, in order in this way to effect a heightenedattentiveness of the driver (quicker detection of critical situations).

A further advantage of the night vision system according to theinvention consists in the possibility of being able to alternateoptionally between two-dimensional and three-dimensional display, or tocombine the two forms of display with one another. Numerous variationsare possible in this case, for example the stereoscopic display can bedeactivated in order to reproduce two-dimensional information at ahigher resolution.

Again, a normal mode of the night vision system could be configured suchthat information (for example driving parameters of speed, rotationalspeed, navigation data, etc) is reproduced as a 2D display, andsituations of the driving path lying ahead are reproduced as a 3Ddisplay.

The night vision system according to the invention offers thefundamental advantage that the driving path lying ahead is perceived bymeans of the three-dimensional display in an accustomed way, that is tosay as under good seeing conditions by day. This reduces the fatiguephenomena which otherwise tend to occur when driving at night. Inaddition, the new system permits the accustomed intuitive reaction(developed from driving experience) to potential danger points with anappropriately shortened reaction time, which contributes to drivingsafety.

Additional information permitting an early reaction to critical drivingsituations can be supplied to the driver by the night vision systemaccording to the invention through the use of extended warningindications (markings or accentuations of objects relevant to safety,displacement into other spatial planes, 2D-3D combinations, etc.).

1. A night vision system for vehicles, having at least two night visioncapable cameras that are fitted on the vehicle at a spacing from oneanother in such a way that a stereoscopic recording is made of thedriving path situated in front of the vehicle, and having means whichcondition the image signals of the night vision capable cameras andreproduce them optically for the driver, wherein means are present thatgenerate for the driver a stereoscopic reproduction of the imagesignals.
 2. The night vision system as claimed in claim 1, wherein themeans for generating a stereoscopic reproduction project imagecomponents from various night vision capable cameras in differentspatial directions.
 3. The night vision system as claimed in claim 1,wherein the means for generating a stereoscopic reproduction haselements that project image components in different spatial directionsby means of optical diffraction (for example diffractive opticalelements, gratings, holographic elements).
 4. The night vision system asclaimed in claim 1, wherein the means for generating a stereoscopicreproduction have elements that project image components in differentspatial directions by means of optical refraction (for example arrays oflongitudinal prisms, microlens arrangements, cylindrical lens arrays orfield lenses).
 5. The night vision system as claimed in claim 1, whereinthe means for generating a stereoscopic reproduction have elements thatproject image components in different spatial directions by means ofreflection (for example retroreflectors, partially reflecting elements).6. The night vision system as claimed in claim 1, wherein the means forgenerating a stereoscopic reproduction have elements that separate imagecomponents optically by means of exclusion (for example barrier grid,color mask) or covering (for example polarization filter).
 7. The nightvision system as claimed in claim 1, wherein the means for generating astereoscopic reproduction have elements that separate image componentsoptically as a function of time.
 8. The night vision system as claimedin claim 1, wherein the means for generating a stereoscopic reproductionhave elements that illuminate image reproduction displays (for exampleLCD displays) in a structured fashion (parallax illumination).
 9. Thenight vision system as claimed in claim 1, wherein the means forgenerating a stereoscopic reproduction have scanning laser systems andelectrooptic modulators.
 10. The night vision system as claimed in claim1, wherein the means for generating a stereoscopic reproduction have atleast two projectors that emit image components in an angularlyselective fashion.
 11. The night vision system as claimed in claim 1,wherein the means for generating a stereoscopic reproduction haveelements (for example lasers, edge-emitting LEDs) that project imagecomponents directly onto the driver's retina.
 12. The night visionsystem as claimed in claim 1, wherein means are present which detect thehead and/or eye position or direction of view of the driver and drivethe means for generating a stereoscopic reproduction as a function ofthe detected head and/or eye position or direction of view.
 13. Thenight vision system as claimed in claim 12, wherein the head and/or eyeposition or direction of view of the driver is detected via cameras,ultrasound or infrared detectors fitted in the interior of the vehicle.14. The night vision system as claimed in claim 12, wherein the meansfor generating a stereoscopic reproduction are driven in such a way thata movement parallax results.
 15. The night vision system as claimed inclaim 1, wherein the means for generating a stereoscopic reproductionhave elements that are worn by the driver (for example polarization orcolor filter spectacles, mini displays, etc.).
 16. The night visionsystem as claimed in claim 1, wherein the means for generating astereoscopic reproduction have swinging or rotating displays.
 17. Thenight vision system as claimed in claim 1, wherein the means forgenerating a stereoscopic reproduction have volumetric displays in whichindividual spatial points are excited to emit light by means of laserradiation, for example.
 18. The night vision system as claimed in claim1, wherein the conditioning of the image signals of the night visioncapable cameras includes the detection and optical accentuation ofobjects in the driving direction.
 19. The night vision system as claimedin claim 18, wherein the optical accentuation of the objects isperformed by marking (for example coloring, contrasting, flashing). 20.The night vision system as claimed in claim 18, wherein the opticalaccentuation of the objects is performed by varying the apparentposition (for example distance) in the stereoscopic reproduction. 21.The night vision system as claimed in claim 1, wherein in addition tothe three-dimensional reproduction image components are also reproducedin a two-dimensional display.
 22. The night vision system as claimed inclaim 21, wherein the driving control parameters (speed, rotationalspeed etc) and/or navigation information (for example GPS) arereproduced as image components in a two-dimensional display.